Skip to content

Fix set-returning functions in the sort key of ColumnarScan - #9981

Merged
svenklemm merged 1 commit into
mainfrom
sven/columnarscan_srf
Jun 11, 2026
Merged

Fix set-returning functions in the sort key of ColumnarScan#9981
svenklemm merged 1 commit into
mainfrom
sven/columnarscan_srf

Conversation

@svenklemm

Copy link
Copy Markdown
Member

Sorting a compressed chunk on a set-returning function failed with
"set-valued function called in context that cannot accept a set".
The function ended up in the sort key pushed down to the per-chunk
sorted scan, but a set-returning function can only be expanded above
the scan. Stop collecting per-chunk sort keys at the first
set-returning function so the sort happens above it.

Fixes #9927

@github-actions

github-actions Bot commented Jun 6, 2026

Copy link
Copy Markdown

@akuzm, @kpan2034: please review this pull request.

Powered by pull-review

@svenklemm
svenklemm force-pushed the sven/columnarscan_srf branch from fb761fb to 80cf0a5 Compare June 6, 2026 11:10
@svenklemm svenklemm changed the title Fix set-returning functions in the sort key on compressed chunks Fix set-returning functions in the sort key of ColumnarScan Jun 6, 2026
@codecov

codecov Bot commented Jun 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Comment thread tsl/src/nodes/columnar_scan/columnar_scan.c Outdated
@svenklemm svenklemm added this to the v2.28.0 milestone Jun 9, 2026
Sorting a compressed chunk on a set-returning function failed with
"set-valued function called in context that cannot accept a set".
The function ended up in the sort key pushed down to the per-chunk
sorted scan, but a set-returning function can only be expanded above
the scan. Stop collecting per-chunk sort keys at the first
set-returning function so the sort happens above it.

Fixes #9927
@svenklemm
svenklemm force-pushed the sven/columnarscan_srf branch from 80cf0a5 to 894474d Compare June 9, 2026 18:01
Comment thread .unreleased/pr_9981
@@ -0,0 +1 @@
Fixes: #9981 Fix set-returning functions in the sort key of ColumnarScan

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Fixes: #9981 Fix set-returning functions in the sort key of ColumnarScan
Fixes: #9927 Fix set-returning functions in the sort key of ColumnarScan

@svenklemm svenklemm Jun 9, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we always link the PR in those and have the PR link to the bugs fixed

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been linking to issues if available, because changelog is user-facing and I think it might be more convenient for users to see the fixed issue right away. The check script allows both.

@svenklemm svenklemm added the force-auto-backport Automatically backport this PR or fix of this issue, even if it's not marked as "bug" label Jun 11, 2026
@svenklemm
svenklemm merged commit cdaf229 into main Jun 11, 2026
63 of 68 checks passed
@svenklemm
svenklemm deleted the sven/columnarscan_srf branch June 11, 2026 07:48
@surister surister mentioned this pull request Jun 15, 2026
surister pushed a commit that referenced this pull request Jun 16, 2026
## 2.28.0 (2026-06-16)

This release contains performance improvements and bug fixes since the
2.27.2 release. We recommend that you upgrade at the next available
opportunity.

**Highlighted features in TimescaleDB v2.28.0**
* **Faster `first()` and `last()` queries on compressed data.**
TimescaleDB derives `first(value, time)` and `last(value, time)`
aggregates straight from the columnstore's batch metadata, skipping
batch decompression entirely. For the "latest reading per series"
lookups that time-series workloads run constantly, that means
meaningfully faster recency queries with no changes to your SQL queries.
* **Lighter, less disruptive continuous aggregate refreshes.**
`refresh_continuous_aggregate()` can now run incrementally in batches —
the same behavior refresh policies already use — enabling breaking large
manual refreshes into smaller chunks (tunable via `buckets_per_batch`,
`max_batches_per_execution`, and `refresh_newest_first`) instead of one
heavy operation. Refreshes also now take a lighter lock while processing
the invalidation log, so they no longer block unrelated concurrent
operations on the same continuous aggregate, improving behavior for
concurrent workloads.
* **Vectorized execution now covers `CASE` expressions.** TimescaleDB's
columnar executor can now evaluate `CASE ... WHEN` expressions directly
on compressed data, so queries using conditional logic stay on the fast
vectorized path instead of falling back to slower row-by-row
decompression. This speeds up a common pattern — conditional
aggregations and computed columns over compressed history — with no
query changes needed.
* **Add new aggregations to a continuous aggregate without rebuilding
it.** You can now run `ALTER MATERIALIZED VIEW <cagg> ADD COLUMN <name>
<type> GENERATED ALWAYS AS (<aggregate>) STORED` to add a new computed
aggregate to an existing continuous aggregate in place — no more
dropping and recreating the whole aggregate just to track one more
metric. New data populates the column going forward, letting your
rollups evolve alongside your application. (Existing rows start as
`NULL`; a forced refresh backfills them when you need historical
values.)

**Deprecation Notice: PostgreSQL 15 Support**
This release marks the final minor version of TimescaleDB that will
support PostgreSQL 15. Starting with our next release, version 2.29.0,
we will officially drop support for Postgres 15, and only support
Postgres 16, 17, and 18; however, all future patch releases within the
current 2.28 version cycle will continue to fully support it. We
recommend planning your PostgreSQL upgrades accordingly to ensure a
smooth transition.

**Deprecation Notice: `chunk_constraint` Catalog Table**
Please note that the `_timescaledb_catalog.chunk_constraint` table has
been dropped and temporarily replaced by a view, which introduces a
change to the underlying objects while maintaining current query
behavior. However, this compatibility view will be completely removed in
a future release. To ensure your queries remain compatible moving
forward, we strongly advise transitioning to the stable contracts
provided by our [informational
views](https://www.tigerdata.com/docs/reference/timescaledb/informational-views).

**Backward-Incompatible Changes**
* [#9934](#9934) Remove
adaptive chunking

**Features**
* [#4054](#4054) Support
`ANALYZE` and `VACUUM` on continuous aggregates by redirecting to the
underlying materialization hypertable
* [#9125](#9125) Increase
the parallelism of `SELECT` queries over compressed hypertables to
approximately match the uncompressed data size
* [#9410](#9410) Mark
`hypertable` and `chunk` as user catalog tables
* [#9416](#9416) Support
some forms of `CASE` expression in columnar aggregation and grouping
* [#9580](#9580) Add
`first` / `last` sparse indexes to compression
* [#9784](#9784) Use
`first` / `last` sparse index for `orderby` metadata on new compressed
chunks
* [#9668](#9668) Allow
database owner to configure hypertables and policies
* [#9701](#9701) Relax lock
during continuous aggregate invalidation log processing
* [#9730](#9730) Add
in-memory observability for compressed chunks
* [#9735](#9735) Improve
`GapFill` row count estimate
* [#9821](#9821) Allow
subquery results which are exec params as GapFill arguments
* [#9825](#9825) Support
`ADD COLUMN` on continuous aggregates
* [#9842](#9842) Suppress
continuous aggregate invalidation tracking during bulk loads
* [#9878](#9878) Remove
`chunk_constraint` catalog tracking for foreign keys
* [#9893](#9893) Remove
`chunk_constraint` catalog tracking for non-dimensional constraints
* [#9903](#9903)
Incremental refresh for `refresh_continuous_aggregate()`
* [#9915](#9915) Remove
`_timescaledb_catalog.chunk_constraint` table
* [#9938](#9938) Add
`rebuild_sparse_index` function
* [#9964](#9964) Add a
function to lock OSM chunk's dimension slice
* [#9980](#9980) Support
`first/last(value, time)` in `ColumnarIndexScan`

**Bugfixes**
* [#9708](#9708) Guard time
bucket parameter handling against bad input
* [#9745](#9745) Check
constraints when adding unique constraints to chunks
* [#9890](#9890) Fix
incremental refresh batch boundaries to align with variable-width
buckets and start only where a chunk and an invalidation overlap
* [#9914](#9914) Fix
use-after-free in segmentwise recompression
* [#9929](#9929) Fix
background jobs being bumped in the queue forever and never running
* [#9955](#9955) Fix wrong
results when using Batch Sorted Merge with no first-last index on a
non-leading order by column
* [#9967](#9967) Block
upgrade after downgrade with first/last indexes present
* [#9976](#9976) Fix wrong
results when comparing a date column to a `timestamptz` value
* [#9977](#9977) Fix `COPY
WHERE` into a hypertable with dropped columns
* [#9981](#9981) Fix
set-returning functions in the sort key of `ColumnarScan`
* [#9982](#9982) Reject
`ALTER TABLE ... INHERIT` when the parent is a hypertable
* [#9984](#9984) Fix
handling of `NOT VALID NOT NULL` constraint for query optimization
* [#9986](#9986) Handle
`MERGE WHEN NOT MATCHED BY SOURCE` on hypertables
* [#9988](#9988) Fix
`time_bucket_gapfill` function detection
* [#10003](#10003) Block
unsafe updates of unique columns on compressed chunks
* [#10024](#10024) Fix
`approximate_row_count` handling of Infinity
* [#10025](#10025) Fix
rename on compressed continuous aggregates
* [#10026](#10026) Fix
chunk skipping near `PG_INT64_MAX`


**New Settings**
* `skip_cagg_invalidation`: skip continuous aggregate invalidation
tracking for DML and DDL in the current session/transaction. Off by
default.
* `stats_max_chunks`: set the per-database compressed chunk statistics
cache capacity. Defaults to 1024 chunks; set to 0 to disable the
feature.

**Thanks**
* @Fabian-2596 for suggesting more accurate GapFill row count estimate
* @otjdiepluong for fixing spelling mistakes in timescaledb source code
comments
* @scimad and @Nosfistis for suggesting expanding coverage for gapfill
arguments
@timescale-automation timescale-automation added the released-2.28.0 Released in 2.28.0 label Jun 16, 2026
@timescale-automation timescale-automation added the released-2.29.0 Released in 2.29.0 label Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backported-2.28.x force-auto-backport Automatically backport this PR or fix of this issue, even if it's not marked as "bug" released-2.28.0 Released in 2.28.0 released-2.29.0 Released in 2.29.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

set-returning functions not handled properly in Columnar Scan

4 participants